In the field of parallel computing system, barrier synchronization is known as a technique of establishing synchronization among a plurality of processes that are performed in parallel processing by a plurality of computing nodes. In the barrier synchronization, a point of synchronization, i.e., a barrier point is set in view of a progress (stage) of the processes. Upon reaching a barrier point in the barrier synchronization, the process is temporarily suspended and the progress of a process at another computing node is awaited. At the moment all the processes performed in a parallel fashion reach the barrier point, an operation of barrier synchronization ends a waiting state, thereby resuming the temporarily suspended process. The synchronization of the parallel processing is thus established among the plurality of processes performed by the plurality of computing nodes.
Depending upon an algorithm in use, a barrier synchronization device in the execution of barrier synchronization may change, on a per stage basis, transmission destinations to which a signal and/or a massage (barrier synchronization message) to the effect that the process has reached the barrier point is to be transmitted. In one barrier synchronization device presented, a transmission destination modification process of the barrier synchronization is implemented using hardware. The barrier synchronization device establishes high-speed barrier synchronization without the intervention of a central processing unit (CPU). Also, the barrier synchronization device includes a synchronization unit that establishes synchronization among a plurality of sets of signals. Even if a plurality of computing nodes are linked to each other via a network, no limitation is imposed on the structure of the network. High-speed barrier synchronization is thus achieved.
A parallel computing apparatus of related art includes a routing controller. The routing controller includes a register storing information relating to a position of the parallel computing apparatus in a barrier synchronization requesting tree. The controller also includes a barrier synchronization control unit performing control in response to a value set in the register. The barrier synchronization control unit transmits a barrier synchronization requesting message when barrier synchronization is ready on its own computing node and on all computing nodes corresponding to routing controllers in its own computing node.
In accordance with an inter-processor data communication method presented, a transmitter side is notified of a physical address of a data receiving region at a receiving side prior to the start of a communication session. The receiver side having double-buffering prevents overwriting on the data receiving region. Partial synchronization with an adjacent computing node is successively performed, leading to automatic establishing of synchronization on all computing nodes. An identifier is attached to the data receiving region, the transmitter side is notified of the identifier prior to the start of a communication session, data with the identifier attached thereto is transmitted to the receiver side, and the receiver side compares the two identifiers. When the two identifiers fail to match each other, an interruption is generated to a processor in the receiver computing node.
The barrier synchronization device in the parallel computing system may perform barrier synchronization successively among a plurality computing nodes. A waiting state to a message relating to a process of a prior barrier synchronization may be canceled by a message to a process of a subsequent barrier synchronization. In other words, there is a possibility that synchronization between the processes is destroyed. It is important that the waiting state to the message relating to the process of the prior barrier synchronization be prevented from being canceled.
It is contemplated that a time interval between a request for a prior barrier synchronization and a request for a subsequent barrier synchronization is set to be long enough. It is also contemplated that a next barrier synchronization is requested in a state that the barrier synchronization of the processes of all computing nodes is complete. A process may receiver earlier a barrier synchronization-established notification that the barrier synchronization of the processes of all computing nodes is complete and may reach a next barrier point. The process still has difficulty in requesting barrier synchronization. It is still difficult to achieve a high-speed barrier synchronization.
It is also contemplated that one computing node may detect the barrier synchronization established on another computing node through a method other than the barrier synchronization. However, since the barrier synchronization is the highest-speed verification method in the parallel computing system. The barrier synchronization is requested to verify the barrier synchronization after all. In other words, a second barrier synchronization is requested to learn the completion of a first barrier synchronization that awaits establishing. As a result, an overhead is caused by introducing double barrier synchronization.
It is an object of the invention to provide a parallel computing system that executes a barrier synchronization process at a high speed while preventing a waiting state of a message of the barrier synchronization from being canceled.
[Patent Document 1] Japanese Laid-open Patent Publication No. 2010-122848
[Patent Document 2] Japanese Laid-open Patent Publication No. 10-049507
[Patent Document 3] Japanese Laid-open Patent Publication No. 06-110845.